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(54) Digital camera with sound recording 

(57) A digital camera 1 for capturing sound passag- 
es as well as still images has the following elements. It 
has a camera apparatus 21 for capturing still Images. It 
also has a sound recording apparatus 101, 102. 103. 
106 for capturing sound passages A memory 107 is 
used for retaining still images as still image files and 
sound passages as sound passage files. The user in- 
terface of the camera is adapted for selectively linking 



one or more still image files with one or more sound pas- 
sage files to form a linked group, and for selectively un- 
linking one or more files from a linked group. This equiv- 
alent treatment of still images and sound passages al- 
lows effective use of the camera for recording ambient 
sound together with images, and for flexible construc- 
tion of sound/image composites representing a user's 
experience in a rich and effective manner. 
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Description 

[0001] The present invention relates to a digital cam- 
era adapted to allow recording of sound as well as still 
images. In particular, the invention relates to a digital 
camera arrangement designed to support the construc- 
tion of different combinations of sound and images. 
[0002] ft is known that the combination of a still image 
and sound is a valuable one for the user, not only where 
the sound is provided as annotation, but also where the 
sound is ambient at the time of recording the still image. 
Examples of systems adapted for recording both still im- 
ages and sound together are disclosed in US Patents 
Nos. 3439598, 4270853, 4270854, 4905029 and 
5276472. A system for display and playback of compos- 
ite sound and still image is set out in European Patent 
Application Publication No. 0827018. This combination 
of sound and still image is also obtainable from certain 
digital video cameras which have capacity to capture 
still images - an example is the Hitachi MP-EGI A. 
[0003] A camera operating system which supports 
augmentation of still images with sound is Digita, pro- 
vided by Flashpoint Technology, Inc. Digita provides a 
menu-driven interlace, and also scripts which allow con- 
struction of composite items at the camera (such as a 
page written in HTML with camera-captured image in- 
serts). 

[0004] In existing products, the sound capture is gen- 
erally optimised for annotation, rather than for recording 
of ambient sound. Accordingly, the microphone Is direct- 
ed towards the user rather than to the imaged scene, 
the sound Is recorded after the image is captured, and 
the combination of media Is limited to one sound clip per 
image. Although capture of ambient sound is a known 
use of cameras with sound capture features, existing ar- 
rangements still provide a simple relationship of asso- 
ciation between a single stiii image and a single piece 
of sound. 

[0005] It is desirable to improve the capture of sound 
together with still Images to allow a rich combination of 
the two which will provide particular benefits to the user 

Summary of the Invention 

[0006] Accordingly, the invention provides a digital 
camera comprising: camera apparatus for capturing still 
images; sound recording apparatus for capturing sound 
passages; a memory for retaining still images as still im- 
age files and sound passages as sound passage files; 
and a user interface adapted for selectively linking one 
or more still image files with one or more sound passage 
files to form a linked group, and for selectively unlinking 
one or more files from a linked group. 
[0007] Still image files and sound passage files may 
be files of different types, in which case selective linking 
between files may be recorded In an index file containing 
all linkage infomnation. Alternatively linked groups may 
have a specific multimedia filetype supporting still Image 



files, sound passage files, and links therebetween: in 
this case, still Images and sound passages may also 
themselves be provided in this filetype, as it possesses 
the inherent capacity to be either a still Image file, a 
5 sound passage file, or both, 

[0008] By treating the images and sound as equiva- 
lent forms of data in this way, rather than merely con- 
sidering a sound passage as an optional addition to a 
still image, richer possibilities for creation and editing of 
10 associated sound and image data become available. 
This equivalence nnakes selection of sound and image 
to be matched together straightfonA^ard. Advantageous- 
ly, it also allows for creation of more complex objects, 
with a linked group containing any of one still Image file 
^5 with one or more sound passage files; one sound pas- 
sage file with one or more still image files; or even a 
plurality of both sound passage files and still image files. 
This fiexibility is particularly important where ambient 
sound is to be captured: for example, a desirable ap- 
20 proach may be to record the sound, and during record- 
ing of the sound to capture a large number of sfill imag- 
es, deciding at a later point which images to keep. With 
a system in which sound is provided through annotation 
of a single image, this desirable mode of operating is 
25 simply not possible. 

[0009] Advantageously, the user interface is adapted 
such that a user can select a first and second view of 
the image files, wherein in the first view a plurality of 
sound passage files, still image files or linked groups 
30 are displayed in an index format for browsing, and 
wherein in the second view a single sound passage file, 
still image file or linked group is displayedfor review and 
editing (including modification of any link). It is helpful if 
In this second view an additional sound passage file, still 
3S image file or linked group Is also displayable, and the 
user interface is adapted to allow linkage between the 
displayed files to form a new linked group. Another de- 
sirable feature is for the sound passage files and still 
image files of a linked group to be viewable, selectively, 
40 either as part of the linked group or independently of the 
linked group. Another useful feature is for the user in- 
terface is adapted so that the sound passage files, the 
still Image files, and the linked objects are all presenta- 
ble to the user as a separate sequence, and that the 
*5 user may select between these separate sequences for 
browsing. This collection of features allows effective 
browsing through a collection of captured data, and ef- 
fective combination and review of the data collected. 
[0010] It is particularly useful, especially in the case 
>o of more complex linked groups, for the digital camera to 
be adapted such that a still Inriage file may be associated 
with a particular point of time or period of time in the 
sound passage of a sound passage file. This is valuable 
If images and sound passages can be captured contem- 
►5 poraneously. In connection with this, it is desirable for 
there to be a means to mark a captured still image or a 
captured sound passage with the time of recording. This 
time stamping can either be absolute, or in the case of 
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contemporaneous recording, relative (that is, an innage 
is stamped with its time of recording relative to the start 
ol the sound passage). Contemporaneous recordings 
of sound and still images may be provided automatically 
as linked groups. This structure allows valuable forms s 
of image association to be created, such as an assem- 
blage of still images appearing in a determined succes- 
sion and spacing against a comnron passage of ambient 
sound. 

[0011] In a development of this approach, embodi- 
ments of a camera may include a further data recording 
device for recording of another data type, and wherein 
the memory device is adapted to record the other data 
type as a further data type file, and wherein the user 
interface is adapted for selectively linking one or more 
further data type files into a linked group, and for selec- 
tively unlinking one or more further data type files from 
a linked group. These further data types may be graphic 
annotation (for example, handwriting input) or video 
clips. 

Description of Figures and Specific Embodiment 

[0012] Specific embodiments of the invention are de- 
scribed below, by way of example, with reference to the 
accompanying drawings, in which: 

Figure 1 shows a system for the capture and repro- 
duction of sound and still images according to an 
arrangement proposed by the present applicant; 

Figures 2a, 2b and 2c show front (image and sound 
capture) side, rear (user interface side) and top 
(capture button) views of a digital camera in accord- 
ance with an embodiment of the invention; 

Figures 3a to 3c show the user interface display of 
Figure 2 in editing mode, and illustrate selection of 
different data elements; 

Figure 4 shows the user interface display of Figure 
2 in editing mode, and shows an audiophoto con- 
sisting of a sound passage with three attached im- 
ages; 

Figures 5a to 5c show the user interface display of 
Figure 2 at three different stages during the playing 
of the audiophoto of Figure 4; 

Figure 6 shows the user interface display of Figure 
2 in editing mode showing an index image of cap- 
tured data; and 

Figures 7a and 7b show the user interface display 
of Figure 2 in capture mode, and illustrate respec- 
tively a viewfinder display and an index display. 

[0013] Basic components in a system for providing an 



image and related sound are shown in Figure 1 . A cam- 
era 1 comprises a conventional digital camera mecha- 
nism for capturing a still image together with a digital 
sound recording device for recording a passage of 
sound. Sound recording means are conventional: a mi- 
crophone 101, an automatic gain controller 102, an an- 
alog/digital converter 1 03 and a codec 1 06 - where play- 
back is also available, as in the embodiment to be de- 
scribed below, a loudspeaker 105 and a digital/analog 
converter 104 are also be provided. A memory 107 for 
storage of audio and image information is also required. 
[0014] It should be noted that although the term "im- 
age" is used here, it does not refer merely to a pictorial 
image. It may also relate to, for example, text captured 
in an image format. Likewise, when the term "camera" 
is used, other devices for capturing a still image can be 
understood to fall within its scope in the context of the 
present invention. 

[0015] Whereas the process of image and sound cap- 
ture may be entirely analog, in which case both an an- 
alog image recording medium (such as film) and an an- 
alog sound recording medium are provided at the cam- 
era, in the context of the present invention digital capture 
of both sound and image is used. Both image and sound 
are digitized on capture and stored in an appropriate 
storage medium (for example on a conventional flash 
memory or an Iomega "Clik" disk ("Clik" is a trade mark 
of Iomega Corporation)). In this event, the digital data 
can be provided to a digital processing means (such as 
personal computer 2). This is advantageous, as it allows 
for easy editing of both image and sound data. The im- 
age data can then be provided from the digital process- 
ing means as a printed image, and the passage of sound 
attached thereto in a digital electronic storage medium, 
such as a flash memory. This is best achieved by means 
of a printer 3. in printer 3. the image is printed in con- 
ventional manner and a sound reproduction device with 
the passage of sound recorded thereon is attached to 
the image. Advantageously, recording and attachment 
of sound is done in the printer itself, but this step may 
be carried out separately (for example, at the sound re- 
cording device 4, as is described further below). Alter- 
natively, it is possible in some embodiments for the im- 
age data and optionally the sound data to be provided 
directly from the camera to the printer (tor example by 
intra red data transfer using technologies currently 
known for infra red printing from personal computers - 
such as under the IrTranP standard): these arrange- 
ments may require the processor in camera 1 to be pro- 
vided with a printer driver for the appropriate printer 
[0016] The result is a tangible representation of the 
image with an electronic storage medium having the 
passage of sound recorded thereon attached to it: this 
is here termed an "audiophoto". To play the passage of 
sound, sound reproduction device 4 is employed. This 
can be connected to the electronic storage medium to 
enable information stored in the electronic storage de- 
vice to be transferred to the sound reproduction device. 
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the sound reproduction device 4 containing means to 
convert the information received from the electronic 
storage device into the passage of sound. However, 
sound reproduction device 4 is adapted to be detacha- 
ble or othenwise remote from the printed image when no 
connection between the electronic storage device and 
the sound reproduction device is required 
[0017] Further details of these system elements are 
provided in copending European Patent Application No. 
98305436.2, the contents of which are incorporated by 
reference herein to the extent permissible by law. A dig- 
ital camera 1 which is an embodiment of the present in- 
vention will now be discussed below, with reference to 
Figures 2a to 2c. 

[001 8] Figure 2a shows the front (image capture) side 
of a digital camera 1 according to an embodiment of the 
present invention. In basic form, digital camera 1 is a 
conventional handheld camera. This camera Is opti- 
mised for capture of ambient sound, rather than anno- 
tation of images, so in addition to a conventional digital 
camera apparatus for capturing still images shown here 
by lens 21 , the front side of the camera 1 also has the 
microphone 101 of a sound recording apparatus (of the 
kind discussed with reference to Figure 1 ) for capturing 
sound passages. In a digital camera optimised for an- 
notation, this microphone 101 would be placed on the 
rear (user interface) side of the digital camera 1 - alter- 
natively, dual microphones, or means to reorient the mi- 
crophone for capture from different directions at differ- 
ent times, may be provided. The captured images and 
sound are eventually stored (in formats and associa- 
tions discussed below in the context of the user interface 
of the digital camera) on a memory for retaining still im- 
ages as still image files and sound passages as sound 
passage files. This memory comprises a removable 
flash memory card 107 held in a slot on the side of the 
camera, and released from the slot by a button release 
22 

[0019] Figure 2b shows the rear (user interface) side 
of the digital camera 1 . The display 23 is (preferably) an 
LCD or other flat screen display. The display can be 
used either for image and sound capture (capture mode) 
or for image and sound editing (editing mode) - both cap- 
ture mode and editing mode are discussed further be- 
low The display 23 is shown in Figure 2b in editing 
mode, with an audiophoio image shown with three dis- 
play elements: image element 231 , sound element 232 
and selection element 233, all of which will be ad- 
dressed below in the discussion of editing mode. Switch 
26 is used to allow the user to toggle between capture 
mode and editing mode. Also provided on the user in- 
terface side of the digital camera is loudspeaker 102, so 
that audio playback is directed towards the user. 
[0020] The user interface controls 24 are provided on 
the user interface side of the digital camera. The user 
interface controls 24 are used to browse, organise and 
edit the captured images and sound, and in particular to 
allow selective linking of one or more still image files 



with one or more sound passage files to form a linked 
group, and for selectively unlinking one or more files 
from a linked group. The specific functions of different 
buttons will be described in greater detail in the discus- 
s sioh of the editing mode, but a brief summary is as fol- 
lows; 



data type selection button 241 allows selection of 
image, sound or both as "active"; 
deletion button 242 allows deletion of the current 
active element; 

linking/unlinking button 243 allows selected ele- 
ments to be linked or unlinked; ^ 
marking button 244 altows a sound clip to be 
marked (for reasons discussed further below): 
forward button 245 allows a user to step fonvard 
throughout data while browsing, or to play a sound 
passage; 

backward button 246 allows a user to step back- 
ward throughout data while browsing; and 
stop button 247 allows a user to stop playing of a 
sound passage. 
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[0021] Figure 2c shows a top view of the camera 1 . 
^5 The top of the camera is where data capture controls 25 
are located. Data capture controls 25 are essentially 
conventional: there is an image capture button 251 eas- 
ily operable with the right hand to record an image, and 
a dual state sound capture start button 252 easily oper- 
30 able with the left hand to begin recording of sound on a 
first button press to end recording of sound on a second 
button press. 

[0022] Editing mode will now be discussed with refer- 
ence to Figures 3 to 6. 
55 [0023] Figure 3a shows an audiophoto in editing 
mode. The audiophoto consists of a still image 231 and 
a sound passage (provided as a graphical representa- 
tion 232 of the sound). The sound passage is marked 
with a tag 31 to indicate the association of an image with 
^0 It - the position of the tag 31 is indicative of the time along 
the sound stream represented by the graphical repre- 
sentation 232. The sound passage is also marked with 
Its duration 32. Selection element 233 indicates in Fig- 
ure 3a that the whole audiophoto is selected. The se- 
45 lection element can be toggled between the states of 
audiophoto/sound element/image element by pressing 
the data type selection button 241 . 
[0024] When the audiophoto is selected as in Figure 
3a, the options provided by the different user interface 
so buttons are as follows. Forward button 245 causes the 
display to advance to, and play, the next audiophoto 
Backward button 246 causes the display to move to, and 
play, the previous audiophoto. Stop button 247 causes 
the sound passage to stop playing. Linking/unlinking 
55 button 243 is for linking or unlinking the displayed ele- 
ments: for an audiophoto, the display at any one time 
will be showing an image element 231 and a sound el- 
ement 232 that are linked to each other. Pressing the 
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linking/unlinking button 243 with an audiophoto selected 
will generally be used to enable the linkage between the 
image element 231 and the sound element 232 to be 
broken: the result could be separated image and sound, 
separated audiophoto and sound, separated audiopho- 
to and image, or even two separated audiophotos (if 
both the image and the sound have additional attach- 
ments) - it can also be used to modify the link. Both de- 
letion and modificatbn of links are discussed below with 
reference to Figure 3b. The deletion button 242 in this 
mode causes deletion of the selected audiophoto (con- 
firmation by pressing a different designated key would 
be required in an effective practical interface), and mark- 
er button 244 is used to mark the sound passage 232, 
as is discussed further below with reference to Figure 
3b. 

[0025] If instead the sound passage is selected, as in 
Figure 3b, then the meaning of the buttons will be dif- 
ferent in some cases. The forward button 245 and the 
backward button 246 now cause display, and play, of the 
next and previous sound p»assage respectively. No 
change is made to the displayed image. Consequently, 
selection of sound passages only can be used to allow 
cycling through sound clips to find a clip for attachment 
to the presently displayed image. Stop button 247 Is 
used to stop playback of a sound clip. Deletion button 
242 is used to delete the selected sound passage. Unk- 
ing/unlinking button 243 is now used essentially for cre- 
ating or removing links between the selected sound and 
the currently displayed image. Pressing the button 243 
causes a link to be established - or if a link already exists 
between the sound passage and the image, for that link 
to be removed. This link may represent a relationship in 
time between the sound passage and the image: in par- 
ticular, for which parts of the sound passage the image 
is to appear. There may be a simple one to one relation- 
ship with no element of time (in which case, the image 
will simply be associated with the beginning of the sound 
passage), but there may be a sequence of images ap- 
pearing successively at different times during the sound 
passage. To support this, the user interface provides 
means on creation of a link to position that link within 
the sound passage. When the link is created, it will ap- 
pear provisionally at the start of the sound passage (un- 
less capture was contemporaneous - see below), and 
can be moved forward or backward along the sound 
passage by use of markers, as described below. In a 
variant of this approach, if capture of images and sound 
are time stamped (either absolutely or if contempora- 
neously captured images and sound are time stamped 
relative to each other), the initial position of a link be- 
tween contemporaneously recorded sound and images 
can be at the actual point of capture of the image within 
the sound record. 

[0026] Use of marking within the sound record can be 
used to improve construction of the relationship be- 
tween images and sound. While a sound passage is 
playing, pressing of the marker button 244 places a 



marker (not shown in Figure 3a or Figure 3b, but visible 
as a visual symbol in sound record 232) in the sound 
record at a time corresponding to the point in the sound 
record at which the marker was pressed. When the 

s sound record is not playing, pressing of the marker but- 
ton 244 places the sound clip into a state in which the 
markers can be used for editing. Initially, a passage in 
the sound record up to the point of the first marker is 
selected. Pressing forward button 245 advances selec- 

10 tion to the first marker itself, and further pressing forward 
button 245 advances selection to the passage of sound 
between the first and second markers. Pressing the but- 
ton again advances to the second marker, and succes- 
sive presses alternate between selection of markers 

1^ and intervening passages until the end of the sound clip 
is reached, at which point the process starts again from 
the beginning of the clip. In this way either a marker 
itself, or the sound passage between two markers, can 
be selected. By use of the backward button 246 it is pos- 

20 sible to cycle through the markers and intervening pas- 
sages In the opposite direction. In this way, a marker or 
intervening passage may be selected and either deleted 
with delete button 242 (and so edit the sound clip itseH ) 
or used to set up a link with an image (by using linking/ 

25 unlinking button 243), the link thus establishing either 
when the image will display within the resulting audio- 
photo (if the link is with a marker) or over what period of 
time within the resulting audiophoto that the image will 
display (if the link is with an interval). 

30 [0027] The alternative to selecting the audiophoto or 
selecting the sound passage is to select the image, and 
this is shown in Figure 3c. The forward button 245 and 
the backward button 246 now cause display of the next 
and previous images respectively. No change is made 

35 to the displayed sound passage. Consequently, selec- 
tion of images only can be used to allow cycling through 
all the images for attachment to the presently displayed 
sound passage. Deletion button 242 is used to delete 
the selected image. Unking/unlinking button 243 is now 

40 used essentially for linking the selected image with the 
currently displayed sound, using principles discussed 
above in relation to Figure 3b. 

[0028] Although not shown here, practical embodi- 
ments will require a confirmation step for significant 
45 events, such as deletion of an audiophoto, image or 
sound clip. Advantageously, an "undelete" function will 
also be provided to allow deletions to be recovered to 
whatever extent is practical. 

[0029] A highly advantageous feature of the embodi- 
50 ment of the present invention is that it supports audio- 
photos that are more than a simple association of a 
sound passage with an image: in particular, an audio- 
photo may comprise a sound passage with one or more 
images, each of which has a relationship in time with the 
55 sound passage, or alternatively a coltection of more than 
one sound [passage with one or more images. An exam- 
ple of an audiophoto with a single sound passage but a 
plurality of images is shown in Figures 4 and 5 Figure 
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4 shows the initial view of such an audiophoto, with the 
audiophoto selected but the sound not playing. There 
are three images 42. 43, 44 associated with the sound 
passage represented by trace 41 , each of which are rep- 
resented in the trace 41 by a discrete clip icon 45, 46, 
47. The clip icons are located in positions corresponding 
to the point in the sound passage at which they are in- 
tended to appear. The change in the display as the 
sound passage plays is shown in Figures 5a to 5c. 
[0030] Clip icon 45 is located at the start of the sound 
passage, so image 42 is displayed from the beginning 
of the sound passage. A visual representation for the 
amount of the sound passage so far played is provided 
by a background bar 48 behind trace 41. In Figure 5a, 
it can be seen that the sound passage is part played, 
but that the second clip icon 46 has not yet been 
reached. In consequence, the first image 42 is being dis- 
played. When the second clip icon 46 position is 
reached, then the second image 43 will be displayed. 
Figure 5b shows display of the second image 43 at a 
point during playback of the sound passage after the 
second clip icon 46 has been reached but before the 
third clip icon 47 has been reached. After the third clip 
icon 47 has been reached, the third image 44 is dis- 
played. In this case, each of the images is associated 
with a point in time (indicating the beginning of display 
of that image) rather than a period of time (as would be 
the case if the image was linked, through use of mark- 
ers, with a part of a sound clip). There is no basic differ- 
ence between the two situations - however, linking with 
passages of time can be used to provide dark intervals 
between pictures, or overlaps. 

[0031] It Is desirable for there to be a view of the var- 
ious available audiophotos and elements which renders 
it easier to compare different elements, or to move be- 
tween elements to be edited. Accordingly the editing 
mode may also provide an index view for viewing mul- 
tiple elements, orcollections of elements. Such an index 
view 61 is illustrated in Figure 6. This index view in- 
cludes all elements that are not part of an existing audi- 
ophoto, together with existing audiophotos. A selection 
frame 62 is provided to allow selection for viewing and 
playing (as in Figure 3a to 3c) of a particular element or 
audiophoto - such selection can be achieved through 
use of forward button 245 and backward button 246. De- 
lete button 242 is used to delete the elements within se- 
lection frame 62. An appropriate button combination is 
used to switch between single audiophoto view and in- 
dex view. When in index view, transition to single audi- 
ophoto view is achieved by pressing data type selection 
button 241 while stop button 247 is being pressed. The 
reverse transition from single audiophoto view to index 
view can be achieved by the same button combination. 
When switching from index view to single audiophoto 
view, the display 23 will change to contain the element 
or elements within the selection frame 62 in single au- 
diophoto view. 

[0032] It is possible from the index view to determine 



whether a particular entry is an image, a sound passage, 
a simple audiophoto or a more complex audiophoto (not 
a simple 1:1 relation of image and sound). Images 64 
and sound passages 65 are readily identifiable as they 
5 contain no sound trace and no image respectively Sim- 
ple audiophotos 63 can be seen to contain one sound 
trace and one image. More complex arrangements, 
such as a single image with two sound clips 66 or a sin- 
gle sound clip with multiple images 67, are again clearly 
10 identifiable as instead of providing a representation in- 
dicative of the start (or any other point) of the sound pas- 
sage, an icon is provided for every element. For exam- 
ple the image 66 with two sound clips has a single image 
shown but two sound traces shown: however, the image 
is 67 with a single sound clip but three images shows all 
three Images in the image view 67, rather than just a 
single image associated with a particular point in time. 
The image view is thus an effective way to detemnine 
quickly the composition (at least the image composition) 
20 of existing audiophotos. 

[0033] Different forms of image view are possible. An 
alternative could be to show alt images, or all sound 
samples, or all audiophotos, and to toggle between 
these alternatives with the data type selection button 
25 241, Another alternative would be to provide separate 
index displays of sound passages and images. Images 
and sound samples could be displayed according to 
their time of capture (particularly appropriate if some 
form of absolute time stamping is available). 
30 [0034] By pressing button 26. the camera is switched 
between editing mode as discussed above, and capture 
mode. The initial image in capture mode will be as for a 
conventional digital camera: a simple image indicating 
what would be recorded if at that point image capture 
36 button 251 were to be pressed. If the image capture but- 
ton 251 is pressed at this point, then a simple image 
results (like images 64 in Figure 6). However, the posi- 
tion changes when recording of a sound passage is in- 
itiated by pressing sound capture button 252. This caus- 
^0 es the display to appear as shown in Figure 7a, with a 
sound bar 71 showing the build up of the sound passage 
(the length of sound so far captured, together with an 
indication that recording is in progress, are also provid- 
ed) and a viewfinder image 72 below. If sound recording 
45 stop button 253 is pressed before image capture button 
251 is pressed, then a pure sound passage will be re- 
corded, like sound passage 65 in Figure 6. However, if 
image capture button 251 is pressed while sound re- 
cording is building up, then a contemporaneous sound 
60 and image capture occurs. Advantageously, this links 
the captured image and the sound passage 71 directly 
with the link registered for initial purposes at the point 
during the sound passage at which the image capture 
occurred (of course, the time relationship in the link can 
55 be changed at a later point through editing). 

[0035] If an audiophoto is built up in this way, it is de- 
sirable for an alternative view to be available to illustrate 
all the elements of the audiophoto. Accordingly, if the 
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data type selection button 241 is pressed during cap- 
ture, then the image display is toggled between the 
viewfinder image 72 shown in Figure 7a to a composite 
of captured image thumbnails 73, each marked with 
their time of recording relative to the start of the sound 5 
passage. The representation of the sound passage 71 
is marked in this view with Indicators 74 showing the 
time of recording for the different images. 
[0036] In addition to pure capture and pure review, it 
is desirable and possible to support an intermediate ar- 
rangement in which new audio or image data can be 
inserted selectively into the existing set, rather than sim- 
ply added to the end. 

[0037] Insertion of such an additional image or sound 
segment is here carried out from the review mode, with 
the appropriate context (audiophoto, sound clip, or im- 
age) for the insertion being selected in the single audi- 
ophoto view on display 23. 

[0038] When the insertion context is a sound clip, the 
user presses image capture button 251 to attach a new 
photo to the clip, or presses sound capture button 252 
twice to capture a new sound passage which is append- 
ed tothe end of the existing sound clip. If the user wishes 
to insert a new sound segment at an appropriate point 
during the existing sound passage, this can be done by 
advancing to and marking with marker button 244 the 
desired insertion point, and in that state capturing the 
new sound clip. 

[0039] When the insertion context Is a still image with- 
out sound, the user can simply press the innage capture 
button 251 to add a new image after the existing image, 
or use the sound capture button 252 to record an audio 
segment which is then attached to the selected image. 
[0040] When the insertion context is an existing audi- 
ophoto, the user can press the image capture button 251 
to attach a new image to the sound clip, creating or ex- 
tending a series of photos for the same clip. The point 
in the sound passage at which the new image Is intro- 
duced can be automatically assigned, or manually po- 
sitioned by use of the marking button 244 as previously 
described. Alternatively, the sound capture button 252 
can be used to record a further sound segment, either 
to append to the audioclip, or to position within the au- 
dioclip by use of the marking button 244 as previously 
described. 

[0041] After capture, the following information Is 
stored in the memory 107: images (stored as discrete 
files), sound passages (stored as discrete files), and 
timestamping and (where relevant) linking information. 
There are two possible approaches for this, either of 
which can be used in satisfactory embodiments. The 
first is for the image and sound to be stored in their own 
separate data types, with all linking, timing and selection 
information stored in a separate index file. This solution 
is probably the simplest to implement, but is not conven- 
ient for selective transfer of information from memory 
1 07 to another device. The secortd approach is for there 
to be a multimedia filetype for audiophotos, supporting 



image, sound, timing and linking information. If such a 
filetype is used, then this filetype can be used for all in- 
formation captured (including images alone, and 
sounds alone) so all files processed by the camera can 
be of the same filetype. This solution allows each result- 
ant audiophoto to be freestanding as a single file, with- 
out reference to an index, rendering selective transfers 
from memory 107 straightforward. This information can 
then be transmitted in any of the paths shown in Figure 
1 : to a personal computer 2 (an effective route for com- 
plex audiophotos - which could then be transmitted, for 
example, by electronic mail), directly to a printer 3 or to 
a reader appliance 4. For simple images, sound pas- 
sages, or audiophotos, sending directly to a printer or 
reader appliance is essentially as described in copend- 
ing European Patent Application No. 98305436.2, but 
differences are required it there are multiple images in 
an audiophoto. For example, a complex audiophoto 
sent directly to a printer could t>e rendered with the vis- 
ual element shown in a composite form as shown in Fig- 
ure 6. The reader appliance 4 could also be replaced 
with a composite reader/viewer appliance (not shown) 
capable of displaying images and playing back sound 
to play back an audiophoto as captured or devised at 
the camera 1 : the functions to be provided need only be 
those offered by the user interface of the camera 1 (pos- 
sibly retaining the browsing features but not the editing 
features), though with preferably sufficiently high quality 
display and sound playback to provide a satisfactory ex- 
perience to the user. 

[0042] In addition to still images and sound, it is pos- 
sible for other data types to be captured at a digital cam- 
era. A digital camera can typically be adapted to capture 
and store short bursts of video without difficulty: such 
video bursts may be provided in the same manner as- 
sound passages as an augmentation of a still image. 
Similarly, means to provide a graphic input at the user 
interface (for example, a pen-based digitiser interface 
and means for recognising handwriting or a custom al- 
phabet such as Graffiti) may be used to provide graphic 
input to provide a similar augmentation. Such augmen- 
tations may be treated in essentially the same way as 
sound passages, and may be used as further (or con- 
ceivably alternative) elements in a linked group. 
[0043] As the skilled man will appreciate, many vari- 
ations upon the features Indicated above can be made 
in devising alternative embodiments of the invention. 
For example, the structural arrangement of the camera 
can be changed considerably - display 23 may be used 
only as a user interface display, with a separate view- 
finder being placed elsewhere on the camera, or a dif- 
ferent arrangement and type of controls can be em- 
ployed for the user interface. 
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camera apparatus for capturing still images; 
sound recording apparatus for capturing sound 
passages; 

a memory for retaining still images as still image 
files and sound passages as sound passage 
files; and 

a user interface adapted for selectively linking 
one or more still image files with one or more 
sound passage files to form a linked group, and 
for selectively unlinking one or more files from 
a linked group. 

>. A digital camera as claimed in claim 1, wherein a 
linked group may consist of one still image file with 
one or more sound passage files. 

A digital camera as claimed in claim 1 , wherein a 
linked group may consist of one sound passage file 
with one or more still image files. 

. A digital camera as claimed in any of claims 1 to 3, 
wherein the user interface is adapted such that a 
user can select a first and second view of the image 
files, wherein in the first view a plurality of sound 
passage files, still image files or linked groups are 
displayed in an index format for review, and wherein 
in the second view an individual sound passage file, 
still image file or linked group is displayed for edit- 
ing. 

A digital camera as claimed in claim 4, wherein in 
the second view an additional sound passage file, 
still image file or linked group is also displayable! 
wherein the user interface is adapted to allow link- 
age between the displayed files to form a new linked 
group. 

A digital camera as claimed In claim 4 or claim 5, 
wherein in the second view a further sound passage 
or still image may be captured and linked automat- 
ically to the individual sound passage file, still image 
file, or linked group displayed. 

A digital camera as claimed in any preceding claim, 
wherein the digital camera is adapted such that a 
still image file may be associated with a particular 
point of time or period of time in the sound passage 
of a sound passage file. 

A digital camera as claimed in any preceding claim, 
wherein the digital camera is adapted such that the 
camera apparatus may capture one or more still im- 
ages at the same time that the sound recording ap- 
paratus is capturing a sound passage. 



group with the sound passage. 

10. A digital camera as claimed in any preceding claim, 
further comprising a time recording means to mark 
5 a captured still image or a captu red sound passage 
with the time of recording. 
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A digital camera as claimed in claim 10 where de- 
pendent on claim 8, wherein the time recording 
means is adapted to mark the time of capture of a 
captured still image relative to a contemporaneous- 
ly recorded sound passage. 



12. A digital camera as claimed in any preceding claim, 
wherein the user interface is adapted for selective 
deletion by a user of a still image file, a sound pas- 
sage file or part of a sound passage file, or a linked 
group. 

13. A digital camera as claimed in any preceding claim, 
wherein the user interface is adapted such that 
sound passage files and still image files of a linked 
group are viewable, selectively either as part of the 
linked group or independently of the linked group. 

14. 
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15. 
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A digital camera as claimed in claim 8. wherein any 
still images captured at the same time as a sound 
passage are provided initially as part of a linked 



A digital camera as claimed in claim 1 3, wherein the 
user interface is adapted so that the sound passage 
files, the still image files, and the linked objects are 
all presentable to the user as a separate sequence, 
and that the user may select between these sepa- 
rate sequences for browsing. 

A digital camera as claimed in any preceding claim, 
wherein the memory retains linked groups as sep- 
arate sound passage files, still image files, and one 
or more index files containing linking information. 

16. A digital camera as claimed in any preceding claim, 
wherein the memory retains linked groups as mul- 

"^0 fimedia files, containing sound passage informa- 
tion, still image information, and linking information. 

17. Adigital camera as claimed in claim 16, wherein the 
sound passage files and the still Image files are both 
multimedia files. 



18. A digital camera as claimed in any preceding claim, 
further comprising a further data recording device 
for recording of another data type, and wherein the 
memory device is adapted to record the other data 
type as a further data type file, and wherein the user 
interface is adapted for selectively linking one or 
more further data type files into a linked group, and 
for selectively unlinking one or more further data 
type files from a linked group. 

19. A digital camera as claimed in claim 18, wherein the 
further data type is handwriting data, and the further 
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data recording device is a device for recording 
handwriting in a digital representation. 

20. A digital canr»era as claimed in clainn 1 8, wherein the 
further data type Is a video clip, and the further data s 
recording device is a device for recording video 
clips, 

21 . A digital camera as claimed in any preceding claim, 
wherein the sound recording apparatus and the io 
camera apparatus are arranged for data capture in 

a substantially common direction. 

22. A digital camera as claimed in any preceding claim, 
wherein the memory is a flash memory device. 

23. A digital camera as claimed in any preceding claim, 
wherein the digital camera is adapted to be hand- 
held by a user for still image and sound passage 
capture. 

24. A program for operating the user interface of a dig- 
ital camera comprising camera apparatus for cap- 
turing still images and sound recording apparatus 

for capturing sound passages, wherein the program ^5 
comprises means for linking one or more still imag- 
es with one or more sound passages to form a 
linked group, and means for unlinking one or more 
elements from a linked group. 
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